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Perhaps the most crucial aspect in gathering information from 

students concerning instru'ction is^ deciding what to ask. ^The kinds of 

items that can be used on student rating instruments can be. analyzed in two 

.ways. One is the focus of the item. - To what particular^ aspiact of the ^ ^ 

student's experience in the course are we directing the student'^s attention?* 

♦ 

It Wght be some segment of course content, a visible instructor behavior, 

or a portion of the course materials. Second, the kitid of it^m includes 

the task that the^tudent is asked to perform when responding to these items. 

Often ,^ the items asK the student to observe what has occurred in the cla^sj^. 

room and make a value judgment or evaluation based upon that observation. 

For example, items such as the following oftetf appfear on student fat4.ng forms: 

"Considering everything j how would, you rate this instructor?' 

"The instructor had an ' adequate knowledge of his^subject matter." 
Or, soxce instrtuments require studen^ts to use a response scali^ that compare^ 
the^ instructor or course to some "average" or Jef^rent group (e.g.. Excellent", 
Above Average, Average, Below Average, Poor). How(ever,j an alternative kind 
of task is a descriptive one rather than an evaluative type. The student's 
task in such items is to observe and descrjbe various compdhei^^s of the 
instructit)ffal setting. An, example of\this type might be to ask the students 

) ' . ■ X 

x^rhether they agree or disagree with the following items: ^ " 

"Instructor communicated at a level appiTopriate to my uriderstandlng. " 
"My work was evaluated in ways that were m^anii^gful to me." 



A brief review of current student opinion forms suggests to us that 



they appear t6 be primarily composed of evaluative tasks concerning instructor*, 
classroom behavior. Recently, much research has been reported concerned with 
the validsition of such instruments. 



*/ . ■ * ■ . 

Some studies have used a criterion- related approach; their hypothesis belag \. 

that if student ^ratings are valid as measures of teaching effectiveness, . 

^. ' ^ » . / 

\ 

they should have a positive correlation with, measures of student performance. 

• ' * ' ' fl . - 

Ihe study by McKeachle , Lin , and Mann (1971) supported thi^ hypothesis, 

using a measure of critical thinking as ai measure of student performance. 

Other researchers (Rodin and Rodin, 1972; Gessner^ 1973; Frey, 1973 j 

, / 

Doyle and IJhitely, 1974)^ hi^^ye used achievement ^m^asures in their v/alidity 

studies, but their results have been conflicting. - 

Other studies have considered the validation of ""such instruments more 

. ' » 

as a matter of . construct validity. If student , ratings ajre valid measures 

o^ .teaching effectiveness, it is usually posited, th^y should not be 

affected Ijy^ various situdent, course, or instructor variables such as 

expected grade, class size*, or instructor's rank which we believe are not 

related to teaching effectiveness. Yet, studies (Costin, Greenough, and ^ 

Menges, 1971) have yielded inconclusive results, either reporting some ' 

moderate c^iwrelations oy no significant correlations among these variableau,,^ 

Howbver, the validity of student rating's using descriptive items appears 

to require\a different focus. Thpugh a description of what has been observed to 

have occurred in an inst^a^tional setting should probably have no relationship 

td a variably such as sex of t^ instructor, •relationships with certain other 

variables might be appropriate. For example, one might expect a positive 



correlation betfwe^ a student's perception that his/her knowledge or skill 
has increased ftom taking a course an4 his/her scor6 on the comprehensive 



tinal examinatioV- One would not necessarily expect, however, to find a 

correlation between the test scores and. a description of the degree to 

which practical ap^licati^s of the material were discussed. * ^ 



4 
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The question then arises concetning what ite^ms ^re appropriate for a 
descriptive ta^k x)f student ratings'K^, in ^th/r wprds, what can students 
be expected to observe? The enqihasis here anould be to focus on those 
aspects of the instructional setting that students are ^ble and willing to 



directly observe and report. 



Could we as faculty legitimately respond, if questioned by the 
, University Regents, concerning whether the President of the University 
has done a good job of representing the University to the legislate ure? 
How are we to know what his best effort might >e? How are we .to knpw what h-e 
or she has actually done? It would be more reasonable, .perhaps, to ask each of us 
if we feel that the President's visible action^ at faculty sena1:e meetings 

♦ T / 

contributed to faculty welfare. Similarly, It would be unreasonable to ask 
students if the instructor has an adequate knowledge of his subject m^t^er. 
Oii what basis^ is the student to judge? Rather we might ask the student to 
describe the presentation of Material.* A, student can 'prpbably observe and 
describe many selected instructor behaviors, course, content and structure, and 
instructional materials and methods. He certainly cannot observe scholarly ^ 

competence, motivation ,;^r many other factors.. Although it "is not frequently 

*■ ■" t ' ' ■ 

r done, we might also ask the stu^gnts to describe their perceptions of the 

I ' 

' outcomes of instruction for them as individuals (e.g.. What h£^.e you 
<^one as a result of the bourse?). The focus should be for students to re- 



spond in t^rms of individual jpercfiption, rather thait 'in terms pf effect on ^ 



the class as a whole. Composite perceptions should be obtained by averaging 
'individual observations, not by averaging (probably iTi|ralid) est;imates of 
the average/perception. As individuals, we could not respond to a question ^ 
on how the entire faculty feels concerning some recent action. Norjshould 

- - ■ ^ > ■ • > • . V : ^ 

fit 
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we expect that a student would always be willing to respond honestly to 
an* item such as: ^« * « , 

"Instructor was willing to' help students outside of class r" 
Each descriptive statemexit should have a focus to which the individual 
student can and will respond validly. 



R^wever, 



The use of descriptive items for student tatings does not, 
eliminate the use of these ratings as partial evidence of teaching effect- 



iveness'. It taerely removes th6 responsibility for evaluation ftom the stu- 
dent and places it onNithose who review the student responses. It becomes 
up to the individual instructor, or his/her colleagues, or the central 
administration to determine the evaluation suggested by the students* per- : 
ceptions, AH this, of course, presumes a satisfactory level of Validity - 
exists in the descriptive- procedure, ' | 

■■ i, 

, V r ' Purpose 

The purpose of this study was to investigate what kin^s of descriptive 
items are or are not related to which course and instructor variables. It 
is possible that there may be di'f£erent relationships between various student 
co^se, and instructor characteristics and the items* used on ^student opinion 
forms according to the fociis of the item. As an example, if students in— 
dlcate that, in general^^^^e course assignments required them to use higher 
level cognitive processes, we should expect to ind that 'student^ also 
describe class activities which encourage th§m to think and be creative 
rather than just reqviire recall of material presented by the instructor. 
Similarly, -iwe might expect that agreement to this type of item would be more 
prevalent in graduate level courses rather than lower-level undergraduate, 
introductory course^ aimed at teaching students basic, factual information. 



In research conducted at the University of Iowa oh an evaluative 
rating' form that asked students to compare the Instructor to the , 

best and worst Instructors they have known, we fdi^d that the rank of 

« " • ■ • ■ ^ ... , ' 

the Instructor was confounded with responses to such items as: 

"Ability to adapt course content to level of students." 

"Skill in presenting material interestingly and clearly." 

However, 4f the focus of the item ji7ere to 6hange such th.at we asked students 

■ - * 

whether or not the ins true tot/ communicated at 'a level appropriate to their 

imde'tstanding or whether or not presentation of material was clear, perhaps 

the rank of the insl:ructor woiiMjK^t be related' to the way in which'Stu-' 

dents respond, nor would other characteristics cif the Instructor, such as 

age or iex. * - 

Researcji into the relationship between students* grades and the items 



on the instrument is, of course, also appropriate. When we are asking 
Students to describe the' course content or the use of instructional materials, 
we might expect that the students* anticipate^ grades should not be related 
t6 the averaged response to these types of items. Yet, we might also ask. 
stt^dents to describe the impact of. the course in terms of whether or not they 
perceive that their kno^^iedge and dicill has been increased or whether thfey ^ 
were motivafted to work beyond mlnimxim requirements. It would not be un- 
reasonable to assume that if students agreed with these statements, they 
might also expect to receive good grades- or that if. the students* perceptions 
are accurate, the instfuctor might actually assign relatively high grades. 

Therefore, iyi thig study we will investigate the relationships between, 
'several course and instructor and ^four types^^olE descriptive items and examine some 
of the issues related to the construct validity of descriptive statements sc0.icit6d 

T / • ; ■ 

from students. 



- Method 
During Fall, 1974, a netw system for solid. ting student opinions about 
teaching called Student Perceptions of Teaching (SPOT) was introduced at tlte 
University of Iova« The system. was developed t> provide a systematic pro- ^ 
cedure for gathering student opinions. The system includes both normative 
data, to assist in interpreting results, and sufficient flexibility^ to adapt 
to the wide diversity of instructional activities at the university. 

Potential items were gathered by revlewiag items froti a variety of 



forms us^d at the University of Iowa and othei 

/ ^ 
chosen /and/pr modified according to the folloir 



institutions* Items wete 
li\g criteria: 1) items were 



included if the:i^e was some empirical evidence' that they retoresented cha^t- 

acterylstlcs on which Instructors and classes differed; 2) all^ items asked 

aboni characterlst\c8 that a student could v^ lidly be expected to observe 

and/ report from his/her 'experiei|ce in the coirse; 3)- items were stated in 

of the Individual student's perception; 4} the item focus reflected a 

iingle characteristic or behavior such that Ithe meaning of the response would 

be easily interpretable;' 5) items were not Highly course specif^ic so. fhat ^ 

^they could be used in mor^ than one course ir department.* Additionally, as 

a group, the itfems were selected to offer considerable t^ariety in, the focus 

of the statements. The pool of 139 descriptive statements "were grouped 

into four types: . • I 

J , I - Course con tenc, '/objectives, & structure 

jl'l - Instructor's behavior 
III ^ Instructional^ methods and materials 
IV - Outcomes of liistruction 
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1 ^ Using an item sampling technique, every f^aculty member at the University 

(N=1344) waj seht^ questionnaire listing a set of 14-19 items. For items 

■ , ' » « • _ • 

of the first three type$, faculty werg asked to indicate whether or "not' each 
item described an "important factor in determining how much students learn" 
in 'any of the courses you teach." For the fourth type of item they were to 
indicate, whether or not the characteristic described represented "an -important^ ' 
outcome °in any of the courses you t^ach." In addition,- for each item they 
indicated whether or not they felt that their "students were competent to 
evaluate- t?hat characteristic (outcome)." Based on a return of 672 questionnaires, 
80 items were chosen for the SPOT, item pool by' selecting primarily those items 
which at least two-thirds of th6 respoadiiig faculty had assessed as being im- 
portant factors or outcomes, and for which at least fifty percent indicated 
that their students were conqpetent to pvaljuiate the characteristic. The items 
were distributed across the four topic areas as follows: IS^items of Type I, 5 
25 items of Type II, 16 Items of Type III, and 21 items of Tyi^ iV. (Appendix A) 

' During the last tjwo weeks of the sprifig semester, 1974, 189 classes 
selected to be representative of courses throughout the unlvers^ity participated 
in a pilot study desigijed to Collect base-line data for. the 80 SPOT items An * 
i|em sampling techniquiAwas used to admini'ster the 80 items in each class to ^ 
enable us to obtain data in all the items without using an undue amount of class 
time. The items were Randomly divided ii/to..)tour groups and each group was 



printed as a separate form. The four forms were then distributed throughout 
each; class in such a |ay that ap|)roximately one- fourth 'of the students answered 
each item, lliese forms were 'accompanied by^a standard set of instructions 
be read to the c^ps. On' the back of each of J:he four forms wete *f our 



additional j^tems. Included to provide part of the data repor,ted In this study. 
(Appendix B) » \ 

Student responses ^to .the 80 SPOT Items utilized a scale which ranged 
from Strongly Disagree (coded as 1) to Strongly Agree (coded as .6) . Indlv^.- 
dual student responses to each statement were summarized as the averaged 
response for each of the 189 classes In the sample. These class means served 

as the dependent variable In all analyses. Elgiht of the Independent variables 

* ^ * - • ^"^^^ ^ ^ 

for this stydy were measures previously reported In conjunction with research 

.? » 

oii student ratings (e.g., Gage, 1961j McDanlel & FeWJausen, 1971). These 
measures are variables whose relationships with ^responses ^to eValuatlve Items 
have Indicated potential difficulties In lntexpref:lng student responses, 

1. Eiq>ected grade for each dass was computed by avet^aglng the ^ 
Individual students' expec^ted grades.. " ' . . 

2. Actual grade assigned whs the average of the individual 

grades actually assigned by. 'the Instructor. ^ , ^ 

3. Whether the course Vais reqU^.red*or ^lictlve^w^Ds computed ad 
'the percent of students liidlcatlng that the course was required 



for them. i 

4. Class size was estimated by the number of students responding 
to 'the SPOT form. r , ' . 

5. Class rank was determined by weighting students reported 
classification (freshman=l, sophomore=2, ...» graduate*5) 

0 

and averaging oyer all students In the couxse. \^ - 

^ Rank of the Instructor was -jaken from University records. 
7. Sex of ^the Instructor was estimated from th^ Instructor's name. 
8^. The nxnnber of^^years since the Instructor had received a bachelor's 

degree was used as a surrogate for age. 

0 n . ' ' 



In addition,t.faur other in^pendent measures we/e used in |ie analyses/ 
These were variables that we had n\t fo'und Reported ia the lUer^ture but 
which we wanted to explore as possibly 'Sing related to some types of It^ns. 

9. . A measure of dif f erence Ibetween the ayerage students' expected ^ 

, ^ • • • grade and the averaged actual grades assigned ^,y the instructor. 

'4 • * • ^ 

. 10. The percent of studeats at,tending on the day of administration 

. of the forms as. compared to the actual enrollment figures for 

' # . \ 

the class. • 
. 11. As a measure relat^to teacher preparatipnAthe rating of the 
graduate ihstitutioii from which the instructoi had received 

• "his/her last degree was b'ased on the Survey of Graduate Education 

• by the American |oraCil on Education (Roose and /^dersen, 1$^ 
Finally, we were interested in how students rated the cognitive c&mplekty - 
of the course according to the taxonomy developed by Bloom et a*, (195^) ' 

and ho«r that perception related to their responses to items concerning . 
mental activity. • • . " 

12. Cognitive complexity was computed as the averaged responses 
of the students to question 4 in Appendix B. 

Results and Discussion 
An^ analysis, of variance was performed for each of the. twelve independent . 
variables with the items nested within type using the averaged student response^ 
to all 80 SPOT Items.. The results of |hose analyses are presented in Tables 

An overall pattem does emerge in the" analyses. Students tend to agree 
most strongly with items describing instructor behaviors and, for most inde- 
pendent variables, the mean Responses by level are more tightly clustered for 



^ * '.- •" - 10 - * • ■ ' ' . ■ J' 
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these items than for the other types of- items. -In cpritrast, eiidor;ement 
by student? is generally least for it^ms concerning putcpmes of instruction, 

the means for tl?e levels, on this type of itenv are typiiaMy most divergent. 
Since our items were not. sampled randomly from pre-specified domains, however, 
it Is not clear whether thlfe resi^lt is > peculiarity of our item sets or a 
more gpneral result with' implications f or instnjment construction ^nd / 
Interpretation of results. Since there was a significant Interadtion b4t«reen 
the.levels' of each of the independent variables ^d the types of items, we 
investigated the relationship between the responses to the individual items 
and each independent >variable. Though a corirelation of +.19 foi\\his sample 
size was Efficient to reject the hypothesis' that-two vdrialjles a're a^correlated 
p<.01, it was decided that a value of +.30 would be of more Vubstantive - . 
i,nt4rest aiid t/ould Serve to focus :attention on only the most substai^ial 
relationships. In the discussion to follow, item? will bfe^ referenced by the 
item number and^t may be necessary to relkc to Appendix A for the coinplete 
text* pf each, item.' ? • ^ . . 



\ ■ ■ . . ' sr. 

Class Size > . - • 

\ 



The significant interaction between class size and type of item indicated 
from examining the means that gtudent^ agrer less strongly with /the items con- 
cerning outcome? of instruction thkn with other types of items /and that overall 
agreement is not as prevalent in larger classes as it "is in-siLller classes 

S&aller classes seemed" to respon^ in a very siiidiar maAtier, p'articularly • 
conceming^rse contene and structure wh^re their, means w/re "Clustered. The ' 
statements concerning outcomes of insitruetion seemg^ to be /^Jeast characteristic 
of larger classes where it would be expected- that students might feel less ' / 
personal impact concerning motivation and interest and (perhaps) learning ^. 



activities. Tljat is, students in smaller classes reported that "instruction "had 

. ■ . ' ' ' . . ^ - ■ ■ ?■ 
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more effect on them than did students in larger claase%. 



' , ' ■ Insert Table 1 about here 

T^hen we look at the individual items we find that one item (#234) had 
a nel^ative correlation (-.42) with this variable. This result is quite 
reasonable because as the size of the class increases, ft^is likely that, 
\ for most classes, l^si^ time would be devoted- to' class discussion. In fact, 
the absence of siic^ a relaticsnship ^ould reflect unfav^abiy on the item's 
' validity. ' - 

Pe.rcent of-^ Students Responding Cong) ay ed to Class. Enrollment ' 

V • ' , - • ' " ' 

We thought that the number pf students who attend class con^ared to 
the actual number^ enrolled might be indicative of students' interest in the 
course and Mghf be reflected Sa. responses^ to the type of Items , concerning 
• interest. Despite the presence of an >interaction between this variable and 
item type, this does not appear to be the case according to the le^el by type 
means. Items relatd^g to outcomes of instruction, many of which concern student 
interest, seem less characteristic of all classes, Th^</group^-"&f -^iS:^^ in 
which 80%'-85% of enrolled students responded to ,the SPOT items seems to account 

for the interaction in this instance. These classes elicited loweir means 

^ .\ — . ' M \ 

(meaning the items were less characteristic) than the other grotips — particularly 



on outcomes of instruction and course structure, iherefore, for some r^son, it 
would appear that outcomes of instruction and^ course structure are most note£tble 
. in course Where' either most: atudents o^ relatively)? few students attend class. 
It is interesting to note here^that, on the average, 81% of the students were 
attending dlasa on the days that the SPOT forms were administered. ^ 



Insert Table 2 about here f^. 
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Class JRaA 

The average class rank df the students showed a significant interaction 
with the type of item^ though there ^did not seem to be substantial differences 
among the levels of this variable! v * ' 

Consistent with eacpedftation is the pattern .of responses for outcomes 
of instruction, Classep primrily consi^pting o$ upperclass students and 
graduate students (group 7) - indicated that the outcomes of Instruction wQ,re 
wore characteristic of their courses than Hid classes of mdstly lower- cAossmen. 

• ^ Insert Table 3 about here ' 



Further analysis ^indicated that only two items of one type, focusing on 
outcomes of instruction, were positively correlated with class rank. The 
relationships with items #427 and #428 indicate the type of activity or moti- 
vation likely to occur in courses tafcen primarily by upperclass and graduate 
students, and, consequently, are compatible with what we presume are common 
teaching practices. ' 

Required vs . Elective Courses 

This analysis produced both a significant interaction effect and a 

significant main effect between classes grouped by percentage of required 

students. Classes which were predominantly required courses for the students 

tendfed to agr^ less with most items thmi did classes which were elected by 

« > 

students. This was p^rticofi^rly -true among classes in which 90% or more of 
the student enrolle^Xas^'^requirement . For these classes, items concerning c,/ 
all but instructor behavior seemed to be particularly less characteristic of 
the instructional environment. That is, the statements in the 4S.tem pool seemed 
to be less descriptive of courses taken as a r^uirement. 
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Insert Table 4 ab.out hera^ 



Lookina-at tfee individual Iten^, three. of the fotlr items (#102, #402, 



VM03, and /i 



1^408) BJtat 



were giost frequently endorsed by student;^ who had 



enrolled in the course primarily as an elective were of^the .saiEe type - 



outcomes of instruction. These results tenjd to confirm the expectation 
that students are^^n^SBk average, morQ interested in courses they *have 
elected to take and more likely^ to enroll in similar courses in thefli ftlture^ 
WHat is, perhaps, of gteater interest i& that responses to items concerning" 
Qpurse cpntent, instructor^ behavior, etc. are tot strongly correlated with 
the degree to which students are required or elect to take courses. 




Sex of Instructor . v * 

^ • ' J 

Th^ significant interaction in this analysis indicated that the items were 

^^omeWhat less characteristic in courses taught by male instructors than female 

instructors, with^the largest difference occurring* on outcomes of instruction. 

However, further analyses did not reveal this variable to have even a moderate 

correlation (.30) with any of the SPOT items. 



Insert Table 5jab out here 



Rank of Instructor 



There was a significant interaction- Wtween the rank of the instructor 
and the student responses to the' types of items. The levfel by type means 



indicated that for classes taught by teaching assistants, thell^te^ ^hceming 
outcomes of instruction tended to be less characteristic than for classes 
taught by faculty, v 
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class practices mid most similar 



All, rmiks oeemed to be similar in 
•in instructor bohavior, but the effect^ of instruction as estimated by ^ 
perceived outcomes seems to be less characteristic for^ teaching assistant 
than" for faculty.. That |Ls ,^ teachdUxg assist^ts appeared ^o have l^ss 
Impact on Self-reported student learning and interest than faculty nietsbers^. 
Again, there we^re no item averages thai: correlated greater tham .30 with 



this, variable.' 




Insert Table 6 about here * . ^ 



Age of Instructor ^ , 

The interaction ^in- this analysis ^etween the agg of the instructor ak 
measured by the v number of years since obtaining a bachelor's degree arfd the" 
t]^es of items indicated a simila;r pattern as with rank of instructpr. Again 
the largest difference in means occurred on iteifts concerning outcome^ of instmc- 
tibn, indicating that less experienced instructors are seen as having less effect 
by students. No correlations greater than .30 emerged in additiofcial^ analysis . 

Insert Table 7 about here 

' • . . f ' ■ ' 

Instructor's Graduate School Training 

For this variable Level 1 represented' instructors whose graduate programs 
were not ranke<f among the beat program^ in their disclj^line in the American 
Council on Education rating. This Analysis indicated that students of th^se 



instructors tended to ag'iree more with all types of items than students o^the 
other Instructors. . . | 
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Of partl-culas interejphere is that in wont other analyses "ins^jtuc tor 
behavior items had the highest means amon^the four types ^Eor all levels. . 
However in tfiis analysis, instructors whose ^raauate programs had the 
highest ratings, had a lower mean than the other grd\ip^and that mean was 
lox^er than the mean for ^course content, whicTi is an atypical pattern. 

■0 

Th^t is, faculty from more "p^estigous" i^stitutione were reported as less 
frequently exhibiting desirable , instructor behaviors than were othel: faculty. 
However, this signl(ficant interaction was not reflected in any correlations 
greater than .30. 



Insert Tal|le 8 about here / 



Average Ejected Grade . ^ o . * 

The significant int^eraction between average' expected grade and types of 
item^ was consistent; concerning this y^riable. StudentjS who ei^ll^ted rela^l^vely 
low gtades; responded that all types of items were less characteristic of 
their courses than th^ other students, and particularly with ;Ltems concerning 
outcomes 6f instruction. Students wh^ expected the highest grades l^greed 
more with^'the items overall. These ^tudeiits responded that , the outcomes of <^ 

' o ■ ^. / ' " . . ' - ^^ 

Instruction were^ particularly more characteristic of their classes with^ a mean 
rating higher than on third type of item which is unusual. Of interest here 
>is 'the XA^ide divergettce amon^he levels for the\ outcome items. Classes who think 
they have done well in a course report that they are more motivated and have 
^learned more. 

_ _ _o_ * 

. Insert Table 9 about' here 
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■ - • .t ■ ' ■ ^ y-. 

In all, 31 of the items showed a positive congelation of .30 or higher 

with average expected gr^de* Close inspection of^ these items reveals that ^ 

" • ?^ • " ■ ■ 

tbe focus <of the item has much to do with th0e ^relationships, 'At one extreme 

*^ 

only two items concerning instructional methods and materials (//326H,and //301) 
correlated with arverage exp^c|ed grade. The first item concerns :t;he degree 
' to which discussion was seen as li^lpful to learning and the latter item 
concepied the students* petception of fairness In grading. 

For the foctp of Instructor's behavior, ,6 it6n» correlate^ with average ' 
expected gra^e (#206, #211, #217, #219, "^#221, and #234). Three of the items 
concWmed the manner in which material was presented arid the other three 
opticemed the, stud^ts* perceptions of an enq)athetic, encouraging Instructor. 

It is apt unlikely that if students agree with these itaasy they expect to 

■ • . " / " /" ■ ' • . ■ ^> ■ 

^ succeslBful in the couyse. 

Eight of the items focusing course cont^t, obJectlveli,'and structure 



Revealed substantial correlatlon$ <With average exjpected grc^de «(#102, #108, ] 
#1143. #116, #122, #127, #128). For thfe most part, they concern helpfulness 
'of cburse element^s and students* level of interest; ^^ctors that might be 
expected to be related to. how i)^ch students think they have ^amed. 

All but six items (#407, #)^Q^, #411, #2!f'l3, #418, and #423) whith are 
concerned with outcomes of instruction are correlated with average expected 
gradi. THIlfS is a logical and desl^fable result, since that^"Eo the extent 
that students^ gain somethiilg from the course ±€ would be reflected in grades. 
In this case, the gains are interest, motivation, confidence, and increas^ 
J Knowledge. " \' ' - ^ 

Because it is accepted practice to aspign higher grades, on the 
average, in upper level courses enrolling primarily jnajors and In graduate 

/ courses, and since our sample' contained many such course^. It seMed that 

f' . • ... 

some of the differences among courses In average expected grade may be due ^ 

' / ■ ■ ' y .IS- ■ ' 
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A' - . • • ■" • ' ' ' 

to that ^factor. Therefore^ ve reanalyzed the relationships between averete 

Item responses 'and average expected graSe adjusting for cla&s rank,* Ot 
the 31' correlations of « 30 or larger between the Items and expected grade » 

pnl^ ,17 still exceeded this value In the new analysis. This ^I'ci^lt suggests 

1^ * * • ■ ' 

that about;^ half of the SPOT Items for which responses are substantially 

I' t> . ^ . 

related to average expected grade appear to reflect prlaarlly coi 



r 

characteristics of upper level courses. 

Average Assigned Grade . * " \^ 

■ ■ ■ . ^\ 
Here again the Interaction^ between the Independent variable and the 

. ' . . 

type of item was an^ expected result. The pattern of^resppnses here was 

similar to the pattern for expected grade, though tTie endorsement .of most 

items among studehts actually assigned lower, grades -was greater than among 

those expecting lower grades. ^ , 

Again, the pattern for sttidents assigned the higher grades indicated 

a higher mean on outcomes of Instruction than on course methods and material^, 

which did not occur for students assigned the lower grades. The means were 

also divergent for the outcome items, s indicataa that instructors 

assign higher grades to ciasse^in which students think tliey have learned 

more and hjave been more . highly motivated. . ' _ ^ vt- • 

. ,W 

Insert Table 10 about here * 

Thire were, however, only 11 items, far fewer.than for expected grades, 
that correlated (.30 or higher) with the actual grades assigned by < the instructs 
or. None of the items concerning instructional metliqds and materials * 
meet fhe . 30 criterion. Of the four items focusing on course content and a 
structure and Instructor's behavior that did meet the criterion (#127, 
#211, #214, #234), all signify that instructors tended to give higher grades t 



As a-part±ai correlation , 



in courses" wfiere students percei 
most of the correlatlons«^appear 

These items (#404, #405, #414, 



where students are motivated to 
the class and have'^an opportunity 



"instructor, relatively higher grades 
« 

Accuracy of Expected Grades 




exchange of icieas occurring. " Again, 
conceihing outcomes of instruction, 

/M27, and #428) suggest that in classes 
Interest beyond what ^curs in 

their knowledge to the 

* i 

r 

gned, . ^ ■ ^ 

C 



A student's perception of his/her peirl 

i . ' ' ' 

differen^ from the instructor's perception 

class ^xpi^cted grade J.n this sample was 3.2 

grade actually 'assi^ed. was 3.0. It seemed 

wK^ther^ there was any irelationshij) between 

oh the part of the students and how they responds 

analysis of variance indicated an. Interaction 



course may be quite 
performance. The average 
system) ; the average ' 

e to investigate 

- • > 

of "misperception" 
he items. ^The 
averaged student 

responses and the accuracy with which stud^ts, as k group, anticipated the 
average grade assignlsd in the course. Looking at the level by type me^'ans, 
it wouXd appear that, wh^re there were many dis'&repancies between expected 
and a^sigi^ed grades, students tended to respond that the items were less 
characteristic of their course on all but the instrucTOr behavior items. 




Insert Table 11 about here 



Cognitive, Complexity • ^ • , ^ 

Using a modified 5-statement summary of Blpom*s taxonomy, the ^students 
were aske^to indicate their perceptions of the cognitive level required by 
the exama and assignments. It was predicted that this variable.would.be 
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related Jt6 st;udents' responses to certain types of items ttiat described 
the cognitive activity that they engaged in during the semester. The 
significant interaction on this variable suggests that some types of items 
are r4felated to the cognitive complexity of- the course. Examining the ievel 
by tjrpe means, indicated in courses designated by] stxident as being of a higher 
cognitive nature, the students tended to 'aigree tubr^with the items overall. 



The different patterns of student Tes|)Qn8es is most distipjguishable With 

items concerning outcomes of instruction. Courses identified by students 

p. ■ • ■ ■ ■ . ■ . 

as ifequifing lower cognitive levels were least characteristic on outcomes 

^ ' " k \ ' 

of insttuction th^ on .other- type^s of items* Of special Interest 

here is the particularlv low mean* for *the group on typd 3 itemis which 

• • ' ^- ' \ ' , / : 

describe^ejthods and iqaterials, including tli.e level of mental activity 

* / ■ ' ; - ' ■ ^ ' , 

required by the exams. For courses x;eauiring: higher cognitive»:4revels, 
the effects of instruction wer^e more' notable. The responses for all 
levels were most similar for items concerning instructor behavior. 



Insert Table 12 atfout here 
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The correlational, analysis of the itenw with the variable on cognitivs 

complexity r'estilts in a striking consistency with our expectationa. Four 
items concerning course content, objectives, and structure (#114j #116, 
#123, //128) show a positive relationship to the independent variable. 
Agreement to each of these items suggests that s'tudents perceived that the 
.structure of the course emphasized the development of higher level thought 
processes. The high^t correlation (.51) of any item with any variable 
in this atudy occurred between item #114 and the queetion on cognitive coaplexltx. 
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The four items focusing oil Instructor's behavior (#203, #21^1, #233, 

#234) that^ere, correlated with cognitive coiq>lexity suggest that in courses 
where there occurs ka exchmig^ of Ideas aaong students and with the Instructor 
.a higher level of cognlt^:ve activity is required. The one itan on Instruc- 
•^Idnal methods (#326)/also concerned wi^h disctission was similarly correlated 
with cognitive complexity; Three other l^ms of this type (#306, #329, #333)^ 
which deal wi^th tUe types of mental activity encouraged by the instructor met 
the cri^rion of this analysis. > , » / . • . {■ 

It is 6f ten -a desirabl^ojutcome of instruction that students are able 
^ to use !md develop upon what they have learned from a course. T|iis type of 
activity is represented in the hierarchy of Bloom's taxonomy. Five items (#409, 
#410, #417, #4;^3, #424) that desctibe the impact of the course as eliciting 
these more involved thought processes from the student were related to the 
question oxr cognitive complexity. 



Summary 

0 . .. 

Of particular interest to us In the data .was that the largest 

differences ^mohg the levels in each of the analyses occurred on items ' 

concerning outcomes of instruction. The other thr^e types of items represent 

in-class practices. Student responses were most similar on these items. If 

any type of item describes the effectiveness of instruction, it is the 

Student-reported outcomes. T^ese were the itema on which classes repreientr 

ing varying levels of selected characteristics differed the most. * 1 ■ 

^ In most cases the expectation that some types of items (depending upon 
their focus) W9uld be related to certain course and instructor characteristics 
was confirmed. .Often where a correlation of +.',3l> occurred between averaged 



Item responses and the variable » the relationship was reaapnabla mttd of tan 




one w&h)id anticipated. Perhaps what Is equally noteworthy In' developing the 

\ i " ' . ' * " * . 

construct validity of this type of -system are those; Items which did not 

t^orrelate with the Independent variables In thld" study. For example , we 

did not. expect and did not find that the class standing of the students was 

'related to any 0f the students' descriptions aif the Instructor's c^ass - 

behavior. ' ^ • 

However y there were some expected ralatlooahli^ that did npt occur. 

An Item such as ^'Adequate tine for questlonl waii provided" vaa as^^Ctad to 

show a latrger negative correlation ^ith clajBa size (r - -.23). There were 

items desirlblng exams^ (#312 and #332) which wa had expectalTto be related 

"t • " 

more strongly to cognitive complexity. (Correlations were .27 and .06.) 

l' 

^ Further analysis is planned 'to enable us to answer some of the.questions 
that we are unable to answer at this time, 'it would be appropriate with 
respect to the^ interactions that occurred,but for which we found no correla-- 
tions,to explore the possibility of nonlinear relationships. 

Additionally, a more thorough analysis of the significant main effects 
(percent required,- expected and assigned grades, and cognitive complexity) is 
planned. ' 
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TABLE 1' 



ANCVA CF SPOT ITEM RESPONSfS BY 



CLASS SIZE 



NO. OF LEVPLS ON INDEPENDENT VARIABLE IS 



LEVEL 


LOWER LIMIT 


UPPER "LI MIT 


^ a 


0, 


- . 10 


.2 


11 


- ' 15 


3 


16 


- 20 ' / 


4' 


21 


30 


5 ' 




- •. 50 


I 6 - 


. 51 


166. 






9 

< 



ANQVA SUMMARY TABLE FOR LEVELS" 



SOLFCE 



-DF 



CLASS SIZE 

^ ■ r 

SUM OF SQUARES 



BET. CLASSES 


188 


3003.63 J 




B 


' 5 


166«38 ^ 




ERR.B 


183 


2837.25 




Will-. CLASSES 


14931. 


7542.88 




TYPE 


3 


106.56 




ITEM Wm TYPE 


76 


1157.38 


. V . ■ 


LEVEL X TYPE 


1'5 


96.00 




I X L WITH.T 


380 


124.94 




^ ERR.W 


14457 


6058.00 




TOT/>L. 


-15119 


10546.50 





MEAN SQUARE 



33.27 
15.50 



35.52 
15.23 

6.40** 

0.3 3 

0.42 



**p<.01 



MEANS 
ITEM TYPES 







1 


2 


n 




ALL TYPES 


LEVEL 


1 


4.88 


4.95 


4.81 


4.72 


4.85 


LEVEL 


2 


4.89 


4.91 


. 4.67 


4*66 


4.79 
















LEVEL 


3 


4*89 


4.95 


4.75 


4.71 


4.83 


LEVEL 


4 


4.79 


"^^^ 


4.61 


'4.56 


4.70 > 


LEVEL 


5 


4.67 


4.73 , 


-4*57 


;4.38 


4.59 


LEVEL 


6 


4.71 


4.71 


4.50 


4.32 


4.57 


ALL LEVELS 


4.82 


4.87 


4.68 


4.60 





.25 



V 



, ' TAbLE ? 



V 



ANCVA CP SPOT ITEH RESPONSfS BY PERCENT RES'PONOIWG/ENROLLEO 
NO. OF LEVEbS ON INO&PENOENT VARIABLE IS 5 



\ 



T 

2 
3 



LOW ERa LIMIT 

, - . 20 . 
A 70 " 

80 
^ 86 
9i 




ANOVA SUMMARY- TABLE FOR LEVELS' 



PERCENT RES PON DlW ENROLLED 



SOtlPCE' 




^ DF 


SUM OF SQUARES 


MEAN SQUARE 


BET.CLASSES 
B ^ 
ERR.B 


" 187 
4 t 
183 


3003.44 
82.63 
2920.81 




20.66 
15.96 


WITH.CLASSES 
TVFE 

ITEM W. TYPE 
' LEVEL X TYPE 
I X L WITH. 7 
ERB.W 

- V 

TOTAL 


148 52 

■ t 

12 
144$7 
^^5039 


7522.13 
i04.ai 
1156.38' 
85.88 
7.13 
6J74.4^ 

idr525.56 


^ ■ ■■■■ 


34.77 / 
15.14 
7.16** 
r 0.02 ' 
0.43 

**p<..01 ' 




1 


MEANS 
ITEM/TYPES 
2 3 4 


ALL TYPES 

0 




LEVEL 1 


4.86 


4. 87 


^r^i'---'- 4.;64 


' ,4.77 


\ 


LEVEL ? 


4. 82 


4.87 


4.61 4.57 


# 4.73 


o 


LEVEL 3 


4.65 


4.73 


4.57 4.37 


4. 59 




LEVEL 4 


4.88 


4.93 


4^75 4.67 


4.82 




LEVEL 5 


4.85 


4.87 


4.69 4.64 


4.77- 




ALL LEVELS 


4.83 


4.87 


4.68 4.60 


1 
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TABLE .3 



5' 



ANOVA CF SPOT ITEM RESPONSES BY CUSS BANK 

NO. OF LEVas ON INDEPENDENT VARIABLE IS 



LEVEL 
1 
2 

3 
4 
5 
6 
7 



LOWER LIMIT 
1.00 
2.50 
3.20 
3.30 
4.00 
4.50 
4.99 



UPPER LIMIT 
2.49 > 
3.19 
3.49 
3.99 • 
4.49 ' 

5.00 „ 



o 

ERIC 



ANCVA SUMMARY TABLE FOR LEVELS' CLASS RANK 
SOL»CE ' OF SUM OF SQUARES 



BET. CLASSES 188 

B > 6 

ERR.B 182 

WITH. CLASSES 14931 

TYPE . 3 

ITEM W. TYPE 76 

LEVEL X TYPE 18 

I X L WITH.T 456 

EBR.W 14378 



3003.63 
\ 189.94 
2813.69 

7542.88 
106.56 

11^7.39 
102. 2M^ 
148.06^. 

6028.63^ 



MEAN SQUARE 



31.66 
15.46 



35.52 
15.23 

5.68** 

0.32 

0.42 



101 fiL 




15119 


.10546.50 








MEANS 










ITEM 


TYPES 






0 

0 


1 


2 


3 


4 


ALL TYPES 


LEVEL 1 


4.66 


4.71 


4.58 


4.34 


4.58 


LEVEL 2 


4.73 


4.80 


4.57 


4.47 


4.65 


level' 3 


4.e4 


4. 89 


4.63 


4.61 


4.75 


LEVEL 4 


4.81 


4.80 


-4.71 


4.60 


4.73 ° 


y 

LEVEL 5 


4.94 


3.03 


4.77 


4.76 


4.89 


LEVEL e 


5.00 


5.00 


4.83 


4.76 


4.90 


LEVEL ^ 


4.90 


4.95 


4.72 


4.75 


4.84 


Vall levels 


4.83 


4.87 


4.68 


4.60 










"27 


1 a 





**p<,01 
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TABLE a 



ANCVA CF SPOT ITEM RESPONSES BY REOUIREO VS. ELECTIVE COURSES 
NO. OF LEVELS ON INtiEPENOENT VARXABL,E IS 5 
LEVEL LOWER LIMIT' UPPER LIMIT 

'° ; 1 0 - 10^ 

2 11 ' ^ 29 . 

3 30 - . 59 

4 60 89 

5 - 90 - 100 

, - .( ' _ . - 

ANOVA SUMMARY TABLE FOR LEVELS' REQUIRED VS. ELECTIVE COURSES 

SOUPCE 

BET.CLASSES 

EBR.B W 

unrt.CL ASSES 
T\FE 
• 'ITBM W. TYPE 
LEm. X TYPE 
I X L hITH.T 
EBR.W 

TOTAL 

MEANS 
ITEM TYPES 



OF 


SUM OF SQUARES 


MEAN SQUARE 


188 


3003.63 




4 


239.56 


59.89** 


184 . 


2764.06 


15.02 


14931 


7542.88 . 




3 


106.56 


35.52 


76 


1157.38 


15.23 ^ 


12 


89.94 • , 


7.49** 


304 


113.94 pj 


0.37 


14536 


6075.0^ 


•0.42 


15119 


10546.50 





**p<.01 







1 


2 


3 


4 


ALL TYPES 


LEVEL 


I ^ 


4.95 


4.98 


4.81 


4.78 


4.89 


LEVEL 


2 


4.88 


4.93 


4.73 


4.68 


4.81 


LEVEL 


3^ 


4.90 


4.94 . 


4.72 


4.. 6 7 


4. 82 


LEVEL 


4 


4.81 

i 


4.81 


4.68 


4.52 


4i71 


LEVEL 


5 


4.60 


4.70 


4.46 


4.34 


4.53 ' 


ALL LEVELS 


4.83 


4.87 


4.68 


4.60 






ff 








^8 
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TABLE 5 



ANCVA CP SPCXUITEM RESPONSES BY SEX OF INSTRUCTOR 

NC, OF LEVELS ON INDEPENDRNT VARIABLE IS 2 

i f L^EVEL LOWER LIMIT UPPER LIMIT 

2 . 2 ^ - 



1 

' 2 



\ 



ANOVA SUWMliRy "TABLE FOR LEVELS- SEX OF INSTRUCTOR 

SOURCE OF SUM OF SQUARES MEAN* SQUARE 



BET.CLASSES . 


~^188 


'3003.63 


B 


1 


24.25 


ERRiB ^ 


187 


2979.. 38 


WITH.CLASSES 


14931 


.7542.88 


TYPE 


3 


1Q6.56 


ITEM h. TYPE 




U157.38 


LEVEL X TYPE 




10.81 


I X L WITH.T 


76 


19.25 


.ERR.W 


1477a 


6248.88 


TOTAL 


15119 


10546.50 



MEANS 




ITEM 

2 


TYPES' 

3 • 


4 


ALL 


TYPES 


4.86 


4.67 


4.58 


4. 


74 


4.93 


4.76 


4.74 


4. 


85 


4. 8,7 


4.68 . 


4.60 







24.2 5 
15.93 



3S.52 
15.2 3 

3.60** 

0.25 

0.42 



**p<.01 



1 



.29 



AMOVA OF SPOT ITEM RESPONSES BY RANK OF INSTRUCTOR 

NO* OF LEVELS ON INDEPENDENT VARIABLE tS 5 

LEVEL LOWER LIMIT UPPER LIMIT ' = 

1,1 - 1. 

2 2 - 2 

.3 3 ^ - 3 

4 Jl - 4 

. ■■ . • .5 , ,5 • - 5 



' ANOVA. SUMMARY TABLE FOR 


LEVELS* RANK 


OF INSTRUCTOR 


SOIRCE 


DF 


% 

SUM OF SQUARES 


^EAN SQUARE 


BET .CLASSES 

ERR^B ^ 


187 
4 

183 

/ 


3000.44 
36.06 
2964.38 


9.02 
16.20 


WITH. CLASSES 

TYPE 

ITEM W. fYPE 
° LEVEL X TYPE 
1 X L WITH.T 
ERR.U 


14852 
3 
76 
12 
304 
14457 


7485.50 
106.75 

1148.44 
75.56 
120^8 

6034.38 


35.58 
15.11 

6.30 ** 

0.40 

0.42 


TOT/IL 

^ 1 " ^ 


15039 


10485.^^4 


**p<.OI 

/ 



MEANS 
ITEM TYPES 



* 


1 


2 


3 


4 


ALLOTYPES, 


LEVEL 1 


4.84 


4.84 


4.71 


4.61 


4.75 

* 


LpVEL 2 


4.87 


4b92 


4.67 


4.66 


4.79 


LEVEL 2 - 


4. £3 


^4.89 


4.70 


4.60 


4.76 


LEVEL' 4 


4,. 7 8 


4.80 


4.63 


4.58 


4.71 


LEVEL 5 


4.71 


4.82 


4.57 


4)42 


4.64 


ALL LEVELS 


4.82 


4.87 


>4.67 


4^^ 
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TABLE 7 



/ 



ANCVA CF SPOT ITEM RESPONSES BY AGE OF INSTRUCTOR 

NQ. OF LEVaS ON INDEPENDENT VARIABLE IS ^ 5 ' 




LOWER LIMH 
I 

5 
11 
16 
26 



UPPER LIMIT, 
'4 
XO 
15 
25 
50 



ANOVA SdMMARY TABLE FOR LEVELS". AGE OF INSTRUCTOR 



SOIPCE 



BET .CLASSES 
.8 
ERR.B 

'WIIH.CLAS^ES 
TYPE 

ITEM W. ITYf^E 
LEVEfT* X TYf^E 
I X L WITH.T 
ERR.W, 

TOT/IL 



LEVEL 1 
LEVEL 2 
LEVEL 3 
LEVEL ^ 
LEVEL 5 
~~ALL LEVELS 



1 

4.71 
4.86 
4.74 
4. £7 
4.86 
4.82 



£>F SUM OF S'adARES 



178 
4 

174 

14141 
3 
76 
12 
304 
p746^ * 

14319 , 

MEANS 
ITEM TYPES 
2 

4. 82 
4.95 
4.81 
4.89- 
4.84 



2a78«;3 
47i.l9 
2830.94 

7145.13 
102. 5p 
1093 .If 5 
81.50 
106.19 
°5761.|l9 

10023*25 



MEAN* SQUARE 



11.80 
16*27 

34.17 
14.39 

6.79 ** 

0.35 

0.42 



& TYPES 




**p<.01 



^.86 



4.68 y 4.59 
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TABLE 8 



ANCVA CF SPOT ITEM RESPONSES BY INSTRUf^OR'S GRADUATE SCHOOL 
NO. OF LEVELS ON IN OEPENDtNf VARIABLE IS 4 



TRAINING 



LEVEL 


LOWER tlMn* 




UPPER LIMIT 


1 


0.0 




0.0 


2 


• 1.0 




X.2 


3 • 


1. 3 • 




1.9 


^4 


2.0 


4 


3.0 



ANCVA SUMMARY TABLE FOR|LEVELS 

SOURCE ^ 

BEl.CLASSES 
B , 
EPP.B 

WITH. CLASSES ' 
TYPE 

I "TEM W. TYPE 
LEVEL X TYPE 
I X L WTH.T 
ERR.W 

TOTAL 



MEANS 
ITEM TYPES 



INSTRUCTOR'S GRApfUATE SCHOOL TRAINING 



OF 


SUM OF SQUARES . 


1£ 


MEAN SQUARE 


104* 


1480.88 


• 






64.56 




21.52 




1416.31 




14.02 ' 


8295 


4531.94 - 






3 


63.44 




21.15 








d.69 




44.13 




4.90** 


22& 


lOi.25 




0.44 


7979 


3662.63 




0.46 , 


8399 


6012. SI 







**p<.01 





1 


2 


3 


*4 ALL TYPES 






LEVEL I 


5.01 


5.10 


4.95 


4.87 • 4.99 






LEVEL 2 


4.73 


4.81 


4.74 


4*49 ,■4/69 






LEVEL 3 


4*74 


4.82 


4.59 


4.50 U.67 \t 


« 


r 


LEVEL 4 


4.81 


4.78 ' 


4.67 


4.59' 4.71 






ALL LEVELS 


4.78 


4.83 


4.70 








. 0 




V 






•0 


0 
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TABLE 9 



ANOVA CF SPOT ITEM RESPONSES BY AVERAGE EXPECTED GRADE 
• NO. OF LEVELS ON INDEPENDENT VARIABLE IS 5 



LEVEL 



LOWER LIMIT 

3,00 
3.20 
3.30 
3.50 



UPPER LIMIT 
2.99 
3. 19 
3.29 
3.49 
4.00^ 



ANOVA SUMMARY TABLC FOR LEVELS- AVERAGj§/^EXPEOTED GRADE 



SOUPCE 

BET.CLASSES 

B 

ERR.B 

WlTHaCL/^SES 
TYPE 

ITEM W. TYPE 
.LEVEL X TYPE 
I X L WITH.T 
EFR.W c 

TOTAL 



LEVEL 1. 
LEVEL ? 
LEVEL 3 
LEVEL 4 
LEVEL 5 
ALL LEVELS 



I 

4. 52 
4.74 
4.90 
4.91 



5.\06 
4.83 



DF 

1 88 
4 

184 

14931 
3 
76 I 
IZ 
304 • 
14W 

15119 



SUM OF SQUaIrES 

3003.63 . 

646.75 
2356.68 

7542.88 
106«56 
: , 1157.38 
; 1P4.06 

117.44 . 
6057.44 

10546.50 



MEANS 
ITEM TYPES 

2 



MEAN SQUARE 



161.69 ** 
12.81 



35.52 
15.23 

8.67 ** 

0*39 

0.42 



**p<.01 



4.55 
4.ff5 
4.85 
4.93 
5.12 
4.87 



3 

4.33 
4.67 
4.70 
4.80 
4.87 
4.68 



4 ALL TYPES 
4.19 4.40 



4.48 
4^63 
4.74 
4.93 
4.60 



4.69 
4.77 
4. 85 
5.01 



4 



* 

33 



TABLE 10 



ANGVA CF SPOT ITEM RESPONSES BY AVERAGE ASSIGNED GRADE 
NO. OF LEVELS ON INOEP€NOENT -VARIABLE IS* 5 



LEVEL 


LOWER LIMIT 


UPPER LIMIT 


1 


1.00 * . 


2.74 


;2 


. 2^75 


2.99 


3, 


3.00 


3.24 




3.25 


- • 3.49 


5 


3.50° 


4.00 



ANOVA SUMMARY TABLE FOR LEVELS' AVERAGE ASSIGNED GR^DE 



SOUPCE , OF 

BET .CLASSES 181 

B ■ ^4 

EPP.8 • 177 

WITH.CLASSES 14378 

T.YPI ' 3 

ITEM W. TYPE 76 

LEVEL X TYPE 12 

I X L WITH.T 304 

Efep.W . 13983 



SUM OF SQUARES 



TOTAL 



14559 



291 8.36 
348.63 
2569.75 

7324.19 
102.25 

1120.25 
9'3.56 
145.94 

5862.19 . 

10242.56 



MEAN SQUARE 

e 

67*16** 
14.52 



34.08 ^ 
14.' 74 

7.80** 

0.48 
.0.42 



MEANS 







1 


ITEM 
2 


TYPES 

3 


4 


ALL TYPES 


LEVEL 


1 


4.62 


4.67 


r4.47 


4.30^ 


4.52 


LEVEl^ 


2 


4.74 , 


4.74 


1 4.64 


4.46 


4.65- , 


LEV^ 


3 


4.89 


4.94 


f.72 


4.67 


4.81 . 


LEVEL 




.4.95 


4.95 


4.78 

c 




4.88 


LEVEL 


5 


4.97 


5.06 


4.83 


4.8,3 


4.94 

• 


ALL LI 


^VELS 


4.83 


4.87 


4.68 


4.60 





**p<.01 



I & 



2'4 



/ 



-33- 



4 



TABLE 11 , 



ANCVA CF SPOT ITEM RESPONSES BY ACCURACY OF EXPECTED GRADE 
NO. OF LEVELS ON INDEPENDENT VARIABLE IS ^ 4 ^ 

LEVEL LOWER LIMIT UPPER LIMIT 
/ ' H ' 0.00 - 0.09 

0.10 ' - -0.19 

3 0.20 * - 0.39 

4 0.40 - 1.11 




/ • 



ANOVA SUMMARY TABLE FOR LEVELS* 



SCfURCE 



OF 



SUM OF SQUARES 



ACCURACY OF EXPECTED GRADE 
HEAN SQUARE 



BET. CLASSES 

B • • 
- EPB.B 


181 

3 

178 




2918.38 
41.56 
2876.81 




4 


WITH. CLASSES 
TYPE 

HEM W. TYPE 
LEVEL X TYPE 
I X L l^ITH.T 
ERR.W 


14378 
3 
76 
9 

228 
14062 


\ 


7324.19 
102.25 

1120.25 
' 85.9.4 
31*38 

€984.38 






TOTAL 




14559 


1024^.56 






• 


1 0 


MEANS 
ITEM TYPES 

2 3 


4 


ALL 


TYPES 


LEVEL I 


4.85 


4.88, 


4.77 


4.70 


4. 


80 


LEVEL 2' 


4.90 


4.92 


• 4.75 


4.62 


4. 


80 


LEVEL 3 


4. 60 


4.89 


4.57 


4.58 


4. 


73 


LEVEL 4 


4.76 


4.79 


4.65 


4.51 


4. 


68 


ALL LEVELS 


4.83 


4.87 


4«68 


4.60 







,13.85 
16.16 



34.08 ? 
14.74/1 

9.55/** 

0.14 

d.43 



,**p<.01 



35 



-34- 



TABLE 12 



ANCVA CF S90J ITEM RESPOfTSES BY COGNITIVE CONPLEXITY 
NO. OF LEVELS ON INOEPENDtNT VARIABLE IS 5 

LEVEL LOWER LIMIT UPPER LIMIT 

,1 1.00 - 1.7a ' 

2 1.76 - . 2.00. 

3 2.01 - 2^.50 ' . 

4 2.51 7 3.00 

5 3.01 " k 5.00 



ANOVA .SUMMARY TABLE FOR LEVELS 
> SOURCE OF 



COGNITIVE COMPLEXITY 



SUM OF SQUARES 



^MEAN SQUARE 



BET. CLASSES 


' 188 


3003.63 


B^ 


4 


356.13 


\^ E P P # B " o 


184 


2647.50 


WITH.CLASSES 


1493^ 


7542.88 


TYPE 




10^.56 


ITEM W. TYPE 


76 


1157.38 


LEVEL )f TYPE 




112.13 


I X L WITH.T 


304 


193.88 


EPP.H 


14536 . 


5972.94 


totaI 


15119 . 


10546.50 



89.03** 
14.39 



35.52 
15.23 

9.34** 

0.64 

0.41 



**p<.qi 



MEANS 
ITEM TYPES 





^ 1 


2 


3 


4 ' 


ALL TYPES 




LEVEL 1 


4.58 


4. 70 


4.37 


' 4.30^ 


4.50 




LEVEL 2 


4.74 


4.77 


4.68 


' 4.45 

/ 

4.63 


^.66 . , ^ 




LEVEL a 


4.86 


4.89 


4.80 


4.80 




LEVEL .'4 


4.94 


4.95 


4.73 


.4. 74. 


4.85 ^ 


\ 


LEVEL 5 


5.04 


5.02 


4.8^ 


4. 89 


4.95 




ALL LEVELS 


4.E3 


4.87 


'4.68 


4.60 
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Appendix A 

I. ^ Course Content, pbjectives, and Structure 

« 

101. Course difficulty was appropriate for my background. 

102. Course organization assisted me in learning. 

103. Subject mattet was intellectually stimulating to me^ 

104. ' Course content was interesting to me. , 
108. Course goals were. clear to me. - • , 

111. I learned basic terms in this area. 

112. Objectives encourageci me to learn the structure and methodology 

of the subject. ' 

114. I was encouraged to apply knowledge and skills in ne^j situations. 

115. Course objectives helped me understand main emphases. 

116. I was encouraged to learn on my own. 

117. Course requirements were clear from the beginning. 

122. Facts and concepts from related fields were presented. 

123. Instructor emphasized ways of solving problems rather than solutions. 

124. Practical applications of the material were discussed. 

127. Adequate time for questions was provided. 

128. ' Instructor emphasized ideas rather than facts. 

129. Rational and intellectual aspects of the subjject wete stressed. 

130. General concepts and ideas were stressed. 

' ■ ~ . - ? ■ ' 

* W 1 



II. Instructor's Behavior 

201, Concepts were presented in a manner that aided my learning, 

203. My work Was evaluated in ways that were meaningful €0 me, ^ 

206, Instructor seemed aware of my needs, abilities , and interests, 

207. Instructor seemed to be^ concerned with*whether t learned the material. 

208. Instructor seemed enthusiastic when presenting coui^se material,, . , 

209. Instructor seemed interested in teaching this course, ^ 

210, Instructor responded to my questions with clarity, 

211, Discussions raised interesting new ideas, 

21^, My questions were answered fully and coD[^)letely , 

214, . Instrjictor was available to me outside of class. 

215, Difficult concepts ^ere explained in a helpful way, 

216, Instructor gave sufficient detail to make generalizations meaningful to me, 

217, Instructor spoke clearly and was easily understood, 

218, Presentations werj interesting and challenging, 

219, Material was summarfized^^in a manner which aided retention, 

221, Instructor commun^.catcd at a^ level appropriate to my understanding. 

226. Instructor summarized major points. 

228. .Instructor made clear what he/she considered important. 

233. Instructor invited criticisms of his/her own 4.4pas. 

234. I was encouraged to participate in class discussion. 

239, Instructor encouraged students to see him/her if they were having difficulty. 

241. ' — festructor discussed points of view other than his/her own. 

242, Recent developments in the field were discussed. ' a 
245, Class presentation seamed well organized. 

247, Well-chosen examples were used to clarify points. 



Ill, Instructional Methods and Materials 



301. 


Grades were based on a fair Jbalance of requirements and content. 




r 

303. 


I knew what improvement was ^ needed from feedback on tests/assignments. 


304. 


Exams reflected the emphases of class presentations. 




305. 
306. 


Exams allowed me to adequately demonstrate what I learned. 

% . .. ^ 

Exams required me to do more than' recall factual information. 




309, 


Exams covered material on which I expected to be tested. 

0 • 




312. 


^ Exams stressed my ability to apply knowledge in new situations. 




315. 


^. . ■ 

Assignments and expectations on homework \^re clear to me. 




317. 


Assignments contributed to ray learning. * 




318. 


Assignments were coni^lstent with course objectives. 






Assigned readings were pertinent to topics presented in class. 




322. 


Cour'sc materials were a helpful* guide to key concepts covered durinig class time 


326. 


ft ' 
Discussion was helpful to my learning. 

& 




329. 


Exams required creative, original thinking. 




332. 


Exaiils required synthesis of jarious parts of the course. 




333. 


Instructor had me apply concep^ts to demonstrate understanding. 


0 




* 





IV. Outcomes of Instruction 



40f. I became more interested 'in ^the subject. 

4 * 

< 

403. - I wps ^^sM^nulated to elect* more courses in this area. 

404. I was stimulated to da additional reading in the area. 

405. I was at^itnulated to discuss new ideas in or out of class. 



407. Ify kno^^dge and skills were increased* 

408. I dey^i(!6ped an appreciation for the subject. 

■■ • Si 

409. ' M7 aMUl in critical thiiiking was increa^sed* 



410. ffyiproblem solving abilities were iiq>roved. 

»411. r learned fundamental principles or theories. 

413. I learned to understand my stre^ths and weaknesses in the area. 

414. Instructor helped'me feel confident in expressing new ideas. 

415. '.I learned to, see relationships among Important topics and ideas. 

416. I was forced to think for myself. 

417. ^ I was motivated to do work beyond minimum requirements. 
'418. I wds motlvate^^ to do my best work. « - 

420. I was stimulated to substantial effort toward learning. 

423. Instructor helped me integrate facts and develop generalizations. 

424. I had an opportunity to demonstrate my knowledge and/or understanding. 

425. I learned new ways to evaluate problems. ^ 

427. I leamedQxow ^o find more information on the subject. 

428, I was motivated to study a topic from thfe^^course on my. own initiative. 



40 



f 



0 



i 



Appendix B 



.-4 



1. What grade do you expect to I 
receive in this course? 





A 


□ 


f 

B 


□ 


C 


□ 


D 


□ 


P 


□ 


F 



2. This course is a(an) 

f ■ 

- n Requirement. 

n Elective. 



3. Class 

^ EIJ freshman ^ 
n Sophomore 
n Junior 
Q Senior 
n Gradtiate 

4. In general » teats /assignments primarily 
reqtiiredf me to (mark only one) 'r^^^ 

n Reproduce facts & opinions. 

I I Integrate concepts. 

I I Apply information in novel 
situations . 

I I Create a product^ plan or approach. 

I I Evaluate the appropriaj&e£es8N>f 
situations. ^ 
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